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Today’s  warfighter  performs  more  complex,  cognitively 
demanding  tasks  than  ever  before.  Despite  the  need  for  more 
extensive  training  to  perform  these  tasks,  acquisition  profes¬ 
sionals  are  often  tasked  to  reduce  training  budgets  and  identify 
optimal  tradeoffs.  Tools  are  available  to  help  them  make  these 
decisions  that  provide  empirical  evidence  of  how  performance 
and  mission  requirements  will  be  affected  by  design  decisions. 

This  article  offers  insights  into  the  utility  of  implementing  a 
Workload  Task  Analysis  (WLTA)  early  in  weapon  systems 
acquisition  for  the  purpose  of  focusing  on  training  system 
decisions,  and  provides  a  description  of  where  WLTA  occurs 
within  the  top-down  functional  analysis  process.  It  concludes 
with  several  examples  of  how  the  WLTA  results  can  be  used 
to  guide  training  development. 
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A  perfect  storm  has  arisen  in  military  training  system  acquisi¬ 
tion.  First,  a  great  deal  of  attention  is  being  given  to  cost  overruns 
in  major  weapon  systems  acquisition  by  the  Department  of  Defense 
(DoD).  As  a  result,  government  officials  are  continually  searching  for 
ways  to  reduce  budgets.  Program  managers  are  continually  being 
asked  to  consider  reducing  training  time  to  reduce  costs,  and  while 
battlefield  technology  is  becoming  more  sophisticated,  it  provides 
much  greater  capability  in  considerably  less  time.  However,  in  many 
instances  the  technology  is  placing  greater  demands  on  warfighters 
by  requiring  a  shift  in  how  work  is  accomplished.  In  many  cases, 
tasks  are  more  sophisticated  and  time-sensitive  than  in  the  past. 
This  shift  in  the  type  of  work  and  speed  of  performance  by  the  war¬ 
fighter  on  the  battlefield  requires  more  training  in  knowledge-based, 
decision-making  tasks  than  ever  before.  This  type  of  training  is  more 
complex  because  it  requires  individuals  to  understand,  integrate, 
and  act  swiftly  on  the  information  generated  from  weapon  systems 
technology;  thus,  additional  time  to  train  warfighters  for  battle  is 
needed  although  budgets  seldom  allow  for  it. 

One  solution  to  this  perfect  storm  is  to  employ  systems  engi¬ 
neering  much  earlier  in  the  acquisition  process  as  advocated  by 
Michael  J.  Sullivan,  Director,  Acquisition  and  Sourcing  Management 
in  testimony  before  the  Panel  on  Defense  Acquisition  Reform,  Com¬ 
mittee  on  Armed  Services,  House  of  Representatives  (Government 
Accountability  Office  [GAO],  2009).  Numerous  dimensions  are 
inherent  to  systems  engineering  in  the  acquisition  process,  one  of 
which  involves  a  systematic  evaluation  of  the  type  of  work  done 
by  a  human  operator  on  a  new  weapon  system  with  the  intent  to 
determine  how  best  to  design  training  solutions  to  support  that 
work.  This  process  is  called  a  Workload  Task  Analysis  (WLTA)  and 
is  incorporated  as  an  element  of  an  overarching  process  called  Top- 
Down  Function  Analysis  (TDFA).  This  methodology  aligns  with  the 
revised  DoD  Instruction  5000.02  (DoD,  2008,  Enel.  8),  which  stipu¬ 
lates  that  "...  where  practicable  and  cost  effective,  system  designs 
shall  minimize  or  eliminate  system  characteristics  that  require  exces¬ 
sive  cognitive,  physical,  or  sensory  skills;  entail  extensive  training  or 
workload-intensive  tasks;  result  in  mission-critical  errors;  or  produce 
safety  or  health  hazards”  (DoD,  2008).  This  article  suggests  that 
if  this  process  is  effective  for  a  weapon  system  design,  it  can  also 
produce  similar  results  for  a  training  system. 

Although  WLTA  is  often  done  iteratively  throughout  the  weapon 
systems  acquisition  process,  this  article  is  limited  to  how  an  early 
analysis  can  initially  identify  areas  where  warfighters  experience  the 
highest  levels  of  workload  that  may  negatively  affect  performance. 
The  findings  from  conducting  WLTA  up  front  have  the  potential  to 
not  only  increase  operator  efficiency  and  effectiveness  by  influ- 
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encing  weapon  systems  design  early  in  the  acquisition  cycle,  but 
may  also  reduce  weapon  systems  life-cycle  costs.  The  design  of 
the  training  system  can  be  included  as  part  of  this  cost  reduction; 
accordingly,  this  article  provides  a  concise  definition,  examples, 
and  insight  into  WLTA  and  the  basic  steps  required  to  perform  a 
credible  WLTA. 


Fundamentals  of  Workload 
Task  Analysis  Methodology 

Modern  technology  has  vastly  changed  the  way  we  do  business 
and  has  improved  our  productivity  by  providing  us  with  many  more 
on-the-job  capabilities.  However,  the  technology  is  only  productive 
when  effectively  employed  by  the  human  operator.  This  effective 
operation  relies  upon  the  cognitive  capacity  of  individuals  com¬ 
bined  with  their  ability  to  operate  the  new  technology  to  its  fullest 
potential.  This  ability  comes  from  a  well-designed  training  program 
that  provides  the  operator  with  pertinent  information  needed  to 
effectively  perform  tasks  on  the  job.  Today,  the  amount  and  speed 
of  information  received  during  war  combined  with  the  complexity 
of  the  technology  that  military  personnel  employ  to  gather  and 
interpret  this  information  further  compounds  workload  burdens 
on  the  individual. 

Historically,  WLTA  has  been  used  to  predict  potential  perfor¬ 
mance  bottlenecks  and  pinpoint  where  to  focus  the  efforts  of 
human  factors  engineers  in  helping  them  make  informed  decisions 
on  new  systems  design.  In  numerous  instances,  operator  workload, 
task  time  demands,  and  interface  design  issues  affected  the  design 
of  numerous  platforms,  from  helicopters  to  airplanes,  ships,  and 
individual  weapon  systems  (Aldrich,  Szabo,  &  Bierbaum,  1989,  pp. 
65-80;  Laughery  &  Corker,  1997).  Recently,  trends  are  re-emerging 
in  military  settings  to  investigate  existing  systems  engineering 
processes  and  procedures  much  earlier  in  the  acquisition  program 
because  of  their  impact  on  reducing  overall  life-cycle  costs  for 
major  weapon  systems  (GAO,  2009). 

Examining  workload  can  assist  analysts  in  determining  the 
degree  to  which  operators  can  successfully  perform  their  job  to 
meet  mission  requirements  (Lysaght  et  al.,  1989).  To  design  a  good 
system,  the  designer  must  comprehend  the  concept  of  workload 
and  understand  what  “optimal”  workload  means  to  performance 
(Mitchell,  2000).  Although  a  number  of  definitions  of  workload  have 
been  provided  over  the  years,  the  overarching  theory  acknowledges 
that  it  is  a  multidimensional  construct  that  considers  the  amount  of 
effort  (e.g.,  sensory,  cognitive,  or  psychomotor)  required  or  invested 
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by  the  individual  in  order  to  perform  on  the  job  (Aldrich  et  al.,  1989; 
Nachreiner,  1995;  Wickens,  1984).  This  performance  is  affected  in 
part  by  (a)  the  demands  of  the  environment  where  the  task  is  per¬ 
formed  (e.g.,  heat,  danger),  (b)  the  complexity  of  the  task  or  the 
system  in  which  the  task  is  performed,  and  (c)  the  capability  of  the 
operator  to  satisfy  those  demands  (Parasuraman  &  Hancock,  2001; 
Parasuraman  &  Rovira,  2005;  Wickens,  2002). 

In  a  WLTA  effort,  insight  into  how  individuals  gather  and  process 
information  about  task  performance  and  the  variables  that  affect 
cognitive  decision  making  (e.g.,  environmental  factors,  task  com¬ 
plexity,  etc.)  can  be  collected  and  analyzed  through  task  network 
simulation  models.  These  models  provide  useful  human  system 
performance  information  to  human  factors  psychologists,  systems 
engineers,  and  instructional  system  developers  to  allow  for  a  myriad 
of  design  trade  decisions.  The  power  of  these  models  lies  in  their 
capability  to  predict  within  a  task  flow  where  and  when  operators 
may  not  be  able  to  perform  specific  tasks  in  a  timely,  effective,  and 
efficient  manner.  This  effectively  assists  analysts  in  determining  tar¬ 
gets  of  opportunity  for  developing  an  optimal  performance  solution. 
An  example  cited  in  a  recent  GAO  report  describes  a  situation  that 
illustrates  where  WLTA  may  affect  the  design  of  weapon  systems. 
The  report  (GAO,  2010)  describes  a  scenario  in  a  surveillance  aircraft 
where  operators  who  are  responsible  for  processing,  exploiting, 
and  disseminating  Intelligence,  Surveillance,  and  Reconnaissance 
(ISR)  data  can  only  use  collected  intelligence  data  if  the  data  are 
visible  to  them. 

Making  ISR  data  discoverable  in  this  way  can  be  accomplished 
through  meta-data  tagging. ..For  example,  a  camera  may  create 
meta-data  for  a  photograph,  such  as  date,  time,  and  lens  set¬ 
tings.  The  photographer  may  add  further  meta-data,  such  as 
the  names  of  the  subjects.  The  process  by  which  information  is 
meta-data  tagged  depends  on  the  technical  capabilities  of  the 
systems  collecting  the  information.  Most  ISR  systems  do  not 
automatically  meta-data  tag  the  ISR  data  when  they  are  trans¬ 
ferred  from  the  sensor  to  the  ground  station  for  processing  and 
exploitation  because  most  of  these  systems  were  developed 
prior  to  DoD’s  emphasis  on  enforcing  meta-data  standards. 
Since  the  sensors  on  these  legacy  systems  are  not  able  to  meta¬ 
data  tag  automatically,  it  is  up  to  each  of  the  military  services 
to  prioritize  the  cataloging  of  the  ISR  data  manually  after  col¬ 
lection.  (p.  5) 

The  solution,  influenced  by  WLTA  analysis  findings,  may  involve 
designing  software  capabilities  that  automate  these  meta-data  tags 
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and  require  the  human  operator  to  confirm  the  accuracy  of  the  tag¬ 
ging.  The  training  implications  resulting  from  this  design  change 
involve  the  identification  of  specific  knowledge  and  skills  needed 
to  operate  this  newly  designed  hardware  and  a  determination  of 
a  training  strategy  for  presenting  this  information  to  the  operator. 
However,  to  fully  understand  where  WLTA  comes  into  play  in  the 
acquisition  process,  it  is  important  to  describe  the  overarching 
analysis  process  involved.  This  is  known  as  the  TDFA. 


Understanding  the  TDFA  Process 

The  initial  assessment  of  any  workload  prediction  methodology 
requires  the  conduct  of  a  comprehensive  mission/task/workload 
analysis  (Aldrich  et  a I.,  1989).  The  TDFA  methodology  is  a  systems 
engineering  approach  that  identifies  mission  requirements  and 
provides  a  comprehensive  capability  for  ensuring  that  the  human 
performance  requirements  are  incorporated  into  the  systems  engi¬ 
neering  process  (Dugger,  Parker,  Winters,  &  Lackie,  1999).  The  intent 
of  the  TDFA  is  to  influence  and  refine  system  design  throughout 
the  acquisition  process.  The  full  TDFA  methodology  used  in  several 
naval  aviation  acquisitions  involves  nine  phases  or  steps  as  shown  in 
Figure  1  (Duke,  Guptill,  Hemenway,  &  Doddridge,  2006).  However, 
only  the  analytical  activity  undertaken  in  the  Mission  Analysis  (Phase 
1.0),  Function  Analysis  (Phase  3.0),  and  Task  Design  and  Analysis 
(Phase  5.0)  will  be  discussed  in  this  article. 


FIGURE  1.  THE  TOP-DOWN  FUNCTION  ANALYSIS  (TDFA)  PROCESS 


1.0 

Mission 

Analysis 


- ► - 

- ► - 

6.0 

[-> 

Interface 

Concepts 

and  Designs 

▼ 

J 

L 

2.0 
Human 
Performance 
Goals 
Analysis 


M 


3.0 

Function 

Analysis 


4.0 

Function 

Allocation 


5.0 

Task  Design 
and  Analysis 


433 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


In  this  TDFA  model,  the  WLTA  is  included  as  the  major  compo¬ 
nent  of  Phase  8.0,  Performance,  Workload,  and  Training  Estimation. 
However,  the  WLTA  cannot  be  undertaken  until  critical  hierarchical 
information  about  the  weapon  system’s  missions,  functions,  and 
tasks  is  available.  In  the  Phase  1.0  (Mission  Analysis),  the  external 
objectives  or  the  “what”  of  the  system  performance  are  identified. 
This  equates  to  the  systems  engineering  “requirements  analysis” 
described  in  American  National  Standards  Institute/Electronic 
Industries  Alliance  632,  Processes  for  Engineering  a  System  (ANSI/ 
EIA,  2003).  System  functions,  which  are  initially  analyzed  in  Phase 
3.0  (Function  Analysis),  describe  “how”  the  system  will  achieve 
performance  requirements.  These  system  functions  are  then  fur¬ 
ther  decomposed  into  human  and  system  tasks,  which  describe  the 
qualitative  and  quantitative  workload  of  individual,  team,  and  crew 
operators  and  maintainers.  This  decomposition  occurs  in  Phase  5.0 
(Task  Design  and  Analysis).  Optimal  design  solutions  based  upon 
recommendations  from  the  task  decomposition  are  integrated 
during  Phase  6.0  (Interface  Concepts  and  Designs)  and  Phase  7.0 
(Crew/Team  Concepts  and  Designs)  to  ensure  system-level  optimi¬ 
zation  and  compatibility.  The  results  of  the  TDFA  are  then  verified 
to  see  if  human  system  integration  is  being  adequately  addressed 
in  meeting  weapon  systems  mission  goals.  Ultimately,  the  process 
provides  a  hierarchy  for  logically  linking  human  performance  (tasks) 
with  the  combatant  commander’s  warfighting  needs  (missions)  as 
shown  in  Figure  2  (Duke  et.al.,  2006). 

FIGURE  2.  MISSIONS-FUNCTIONS-TASKS  HIERARCHY 
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The  WLTA,  which  occurs  in  Phase  8.0  (Performance,  Workload, 
and  Training  Estimation),  uses  information  obtained  in  the  previously 
mentioned  TDFA  phases.  To  gain  an  appreciation  of  the  WLTA,  a 
brief  description  of  the  mission,  function,  and  task  analysis  phases 
of  the  TDFA  model  is  provided. 

Mission  Analysis  (Phase  1.0) 

The  Mission  Analysis  Phase  of  the  TDFA  serves  to  determine 
and  document  the  overall  purpose,  objectives,  and  mission  require¬ 
ments  of  a  weapon  system.  Initially,  a  weapon  system’s  primary  and 
secondary  missions  (e.g.,  Anti-Submarine  Warfare,  Anti-Surface 
Warfare,  etc.)  are  determined  and  correlated  with  system  mis¬ 
sion  tasks.  The  Universal  Navy  Task  List  or  other  Service  task  lists 
provide  the  basis  for  identifying  the  system  mission  tasks.  Initial 
metrics  are  also  established  to  measure  results  in  the  execution  of 
missions.  This  involves  creating  Measures  of  Effectiveness  (MOEs) 
and  Measures  of  Performance  (MOPs),  which  are  used  to  determine 
the  system’s  ability  to  support  the  achievement  of  an  operational 
mission  and  the  technical  performance  standards  that  a  system 
must  achieve  to  satisfy  the  MOEs,  respectively  (Chairman  of  the 
Joint  Chiefs  of  Staff,  2003).  MOPs  also  serve  as  high-level  standards 
by  which  many  systems  operators  will  be  evaluated. 

The  system  constraints  and  boundaries,  which  may  have  an 
impact  on  training  program  design,  are  also  identified  in  the  Mis¬ 
sion  Analysis  Phase.  For  example,  in  dealing  with  the  acquisition  of 
a  training  system  for  a  new  surveillance  aircraft,  one  must  determine 
if  the  scope  of  the  platform  (system)  includes  a  ground  station.  If  it 
does,  then  the  training  analysis  must  consider  the  infrastructure  and 
all  associated  logistics  associated  with  the  ground  station  as  well  as 
that  of  the  platform.  Information  about  constraints  and  boundaries 
is  usually  obtained  by  analyzing  its  high-level  mission  objectives, 
which  are  found  in  acquisition-related  publications  such  as  the 
weapon  system’s  Initial  Capabilities  Document,  Capability  Develop¬ 
ment  Document,  Performance  Based  Specification,  and  Office  of 
the  Chief  of  Naval  Operations  Instruction  1000. 16K  (2007).  Once 
the  overall  purpose,  objectives,  and  mission  requirements  are  deter¬ 
mined,  it  must  be  determined  what  the  system  must  do  to  satisfy 
the  mission  requirements.  This  analysis  is  called  a  Function  Analysis. 

Function  Analysis  (Phase  3.0) 

The  goal  of  the  Function  Analysis  Phase  is  to  define  performance 
at  the  level  of  detail  where  it  is  possible  to  design  all  subsystems 
or  components  needed  to  satisfy  performance  requirements.  For 
example,  if  a  mission  requirement  exists  for  weapon  systems  to  per¬ 
form  surveillance,  then  the  weapon  system  (e.g.,  an  aircraft)  must  be 
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designed  with  a  means  to  undertake  the  surveillance  function  (e.g., 
must  have  a  radar  system).  Functions  provide  the  means  to  align 
mission  requirements  to  the  specific  system  hardware  and  software. 
The  Department  of  Defense  Architecture  Framework  (DODAF) 
data  model  views,  which  are  developed  by  mission  system  engi¬ 
neers,  provide  detailed  hardware  and  software  information  about 
the  functions  performed  by  equipment  embedded  in  the  technical 
systems  (e.g.,  radar  system)  comprising  a  weapon  system  (DoD, 
2009).  They  also  supply  important  information  about  the  technical 
communication  between  the  equipment  (e.g.,  describe  technical 
cues  and  responses  of  hardware  and  software  tasks).  The  integrated 
product  team  (IPT)  training  team  must  train  the  human  operator  on 
how  to  effectively  interface  with  the  equipment  and  must  ensure 
the  operator  understands  how  technical  activities  accomplish  mis¬ 
sion  requirements. 

Task  Design  and  Analysis  (Phase  5.0) 

In  the  Task  Design  and  Analysis  Phase,  analysts  develop  initial 
tasks  that  describe  how  humans  will  perform  assigned  system  func¬ 
tions.  During  this  phase,  hardware  and  software  tasks  are  linked  with 
human  performance.  Information  in  this  step  usually  comes  from  a 
stratified  sample  of  subject  matter  experts  (SMEs),  who  either  have 
performed  the  tasks  or  are  very  familiar  with  how  the  tasks  should 
be  performed.  One  way  to  collect  task  information  is  to  use  critical- 
event  scenarios  where  SMEs,  using  a  flowchart,  identify,  describe, 
and  document  the  individual  tasks  and  subtasks  they  perform  at 
their  workstation  during  a  mission.  Each  scenario  should  depict  a 
unique  mission  area,  which  allows  the  SMEs  to  collect  information 
about  the  different  types  of  workstation  tasks  performed  dur¬ 
ing  different  missions.  The  authors  recommend  that  all  technical 
publications  and  applicable  reference  workstation  documents  are 
made  available  to  SMEs  during  this  exercise.  In  cases  where  the 
SMEs  are  providing  information  based  on  their  experiences  with 
a  legacy  system,  the  legacy  tasks  are  compared  to  the  high-level 
notional  functions,  missions,  and  tasks  obtained  from  the  DoDAF 
model  views  and  documented  in  the  TDFA.  As  the  new  systems  on 
the  weapon  system  are  developed,  changes  will  be  assessed  and 
the  functional  architecture  will  be  modified. 


Undertaking  a  Workload  Task  Analysis 

As  stated  previously,  a  WLTA  effort  provides  insight  into  an 
operator’s  perceived  level  of  effort  to  complete  a  task  and  the 
variables  that  affect  decision  making.  The  WLTA  is  conducted  in 
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Phase  8.0  of  the  TDFA  process.  The  information  from  the  Mission, 
Function,  and  Task  Design  and  Analysis  Phases  enables  the  WLTA 
analyst  to  understand  the  workload  associated  with  a  given  task/ 
mission.  The  workload  activity  is  divided  into  time  and  information 
processing  demands  (e.g.,  visual,  auditory,  cognitive,  and  psycho¬ 
motor  [VACP])  so  it  can  be  examined  from  various  perspectives. 
These  perspectives  assist  the  human  factors  psychologists,  systems 
engineers,  and  instructional  system  developers  to  better  analyze 
high-workload-demand  tasks.  The  discussion  that  follows  contains 
a  brief  description  of  the  two  types  of  workload  data  collected  (i.e., 
time  estimation,  information  processing  estimation),  and  provides 
examples  of  potential  training-related  performance  solutions  to 
workload.  These  training  solutions  are  not  intended  to  be  all-inclu¬ 
sive,  but  rather  provide  a  starting-off  point  for  any  IPT  training  team 
to  consider  for  its  own  platform. 

Time  Estimation 

In  this  portion  of  the  WLTA,  the  analyst  is  interested  in  the  time 
spent  on  particular  components  of  the  overall  mission.  The  results 
from  this  analysis  can  be  an  indication  of  the  complexity  of  the 
task  or  performance  inefficiencies  (e.g.,  poor  system  design,  lack 
of  training,  etc.).  Depending  on  the  analyst’s  interpretation  of  the 
cause  of  the  time  spent  on  certain  components  of  an  overall  mission, 
targeted  training-related  solutions  may  be  identified. 

Several  options  are  available  to  gather  time-estimate  data.  Ide¬ 
ally,  the  collection  of  time  data  should  come  from  direct  observation 
of  actual  operators  performing  the  tasks  during  a  live  mission  or  on 
a  simulator  during  a  training  exercise.  In  many  cases,  however,  this  is 
not  possible,  especially  if  the  weapon  system  has  not  been  built  or 
the  nature  of  the  tasks  do  not  allow  for  direct  observation  during  a 
mission.  As  an  alternative  to  observation,  domain  SMEs  provide  esti¬ 
mates  of  the  amount  of  time  spent  performing  each  task,  as  well  as 
whether  each  task  is  continuous  (no  observable  start  or  end  point) 
or  discrete  (actions  with  definite,  observable  start-end  points).  Prior 
to  the  SMEs  providing  these  ratings,  they  are  provided  examples 
of  discrete  (e.g.,  manipulating  a  knob  on  a  computer  console) 
and  continuous  (e.g.,  monitoring  targets  on  a  radar  screen)  tasks. 
These  tasks  are  graphically  illustrated  on  flowcharts  that  clearly 
depict  what  is  actually  being  done  by  an  operator  in  response  to  a 
specific  cue.  The  flowcharts  make  it  easier  to  estimate  the  time  it 
takes  to  accomplish  a  specific  task,  and  provide  analysts  with  both 
an  average  time  on  each  discrete  task  and  a  range  of  time  for  the 
continuous  tasks. 
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Information  Processing  Estimation 

The  next  step  of  the  WLTA  is  to  determine  and  categorize  the 
amount  of  workload  required  to  perform  the  task  during  a  typi¬ 
cal  mission.  This  step  is  likely  the  most  challenging  as  it  requires 
the  analyst  to  implement  a  modeling  approach  that  accounts  for 
tasks  occurring  simultaneously,  the  types  of  tasks  being  done,  and 
other  factors  that  may  shape  the  performance  of  the  operator. 
Research  suggests  that  humans  are  able  to  process  information 
across  multiple  different  VACP  channels,  as  illustrated  in  Christo¬ 
pher  D.  Wickens’  Multiple  Resource  Theory  model  (Wickens,  2002; 
Wickens,  Sandry,  &  Vidulich,  1983).  In  such  cases,  the  summative 
workload  demands  of  multiple,  simultaneous  tasks  on  one  channel 
can  provide  some  indication  of  the  likelihood  an  individual  would 
be  able  to  perform  two  or  more  tasks  at  the  same  time  with  a  given 
workload.  For  example,  if  a  task  calls  for  an  operator  to  simultane¬ 
ously  aim  a  weapon  at  a  target  (rated  a  4  on  the  visual  channel  by 
the  operator)  and  make  some  fine  discrimination  of  symbols  on  a 
screen  (rated  a  5  on  the  visual  channel),  the  combined  demand  on 
the  operator’s  visual  channel  would  exceed  the  highest  rating  pos¬ 
sible  on  the  visual  scale  (see  Appendix).  This  high-workload  rating 
on  the  visual  channel  would  be  cause  for  concern  in  a  design  that 
required  an  operator  to  simultaneously  perform  these  two  tasks. 
The  VACP  scales  are  provided  in  the  Appendix. 


FIGURE  3.  RESULTS  OF  A  VCAP  ANALYSIS  DONE  TO  • - Visual 

ILLUSTRATE  OPERATOR  PERFORMANCE  DURING  A  Auditory 
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^ - Number 
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From  a  training  analyst  perspective,  it  is  necessary  to  assess  the 
workload  demands  on  an  operator  at  different  intervals  throughout 
the  mission.  To  do  so,  the  VACP  components  of  the  tasks  have  to 
be  estimated  (McCracken  &  Aldrich,  1984;  Szabo  &  Bierbaum,  1986) 
and  populated  into  a  discrete-event  simulation/modeling  software 
tool.  A  number  of  commercial  and  government  discrete-event 
simulation/modeling  software  tools  (e.g.,  MicroSaint,  Improved 
Performance  Research  Integration  Tool  [IMPRINT]),  are  available  to 
provide  the  capability  to  account  for  operator  ability  to  multitask 
across  noncompeting  processing  channels  consistent  with  Wickens’ 
Multiple  Resource  Theory  (Wickens,  2002;  U.S.  Army  Research 
Laboratory,  2005).  Figure  3  provides  an  example  of  a  simulation 
showing  how  operators  reacted  to  a  surveillance  mission  situa¬ 
tion.  It  uses  the  dynamic  modeling  technique  of  Coloured  PetriNet 
(Kristensen,  Christensen,  &  Jensen,  1998)  to  illustrate  the  predicted 
VACP  demands  on  an  electronic  intelligence  (ELINT)  operator  in  a 
surveillance  aircraft  while  performing  assigned  tasks  in  a  specified 
period  of  time  during  an  operational  mission.  Careful  evaluation  of 
these  workload  predictions  provided  the  analyst  with  insight  on 
candidate  tasks  where  workload  demands  may  be  improved.  For 
example  in  Figure  3,  the  data  indicate  that  about  7  minutes  into  the 
mission,  the  operator  had  to  undertake  several  tasks  while  respond¬ 
ing  to  target  cues  at  the  workstation.  During  30-second  intervals, 
the  operator  had  to  simultaneously  employ  several  skills,  which 
caused  a  temporary  visual  and  cognitive  overload.  The  operator 
used  high  visual  skills  (shown  by  blue  line  showing  8.5— exceeding 
the  visual  workload  scale).  Thus,  the  operator  may  not  have  been 
able  to  “see”  all  the  target  data  available  on  the  screen  during  that 
30-second  interval.  The  operator  also  used  high  cognitive  skills 
to  interpret  what  was  being  seen  and  heard  (shown  by  green  line 
showing  8.3— exceeding  the  cognitive  workload  scale).  Thus,  the 
operator  may  not  have  been  able  to  comprehend  the  information 
presented.  Both  of  these  were  done  while  interpreting  sound  pat¬ 
terns  (shown  by  red  line  indicating  7.0  on  the  auditory  scale)  and 
manually  adjusting  a  thumbwheel  (shown  by  the  light  blue  line  indi¬ 
cating  5.8  on  the  psychomotor  scale).  This  “task-stacking”  situation 
resulted  in  the  operator  exceeding  visual  and  cognitive  capacity  for 
approximately  30  seconds,  meaning  critical  information  may  have 
been  missed,  which  could  impact  overall  mission  performance.  With 
this  information,  the  engineers,  human  factors  psychologists,  and 
instructional  system  designers  can  begin  to  develop  alternatives  for 
task  redesign,  human  engineering  improvements,  and/or  training 
solutions.  In  the  following  section,  a  few  examples  of  this  process 
are  provided. 
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Workload  Solution  Identification 

The  risk  in  developing  weapon  systems  without  significant 
consideration  for  how  the  operator  will  actually  utilize  them  is  that 
the  system  utilization  may  fall  significantly  below  its  potential. 
Additionally,  a  great  deal  of  time  and  resources  may  be  required  in 
developing  training  systems  for  the  operator.  Utilizing  WLTA  data 
up  front  may  prevent  this  situation.  Workload  solutions  can  take 
many  forms,  and  should  be  based  on  the  cost,  schedule,  and  per¬ 
formance  considerations  by  the  weapon  systems  program.  The  final 
solution(s)  chosen  should  be  guided,  at  least  in  part,  by  the  results 
from  the  WLTA.  For  purposes  of  this  article,  only  training  solutions 
are  discussed,  but  WLTA  identifies  where  workload  issues  may  arise 
within  a  mission  scenario  for  operators  of  a  system  and  narrows  the 
focus  to  particular  tasks  and  combinations  of  tasks. 

WLTA  provides  information  about  the  operator  tasks  and  pro¬ 
vides  the  training  developer  with  data  about  how  user  interfaces 
are  structured  to  enable  performers  to  effectively  use  the  weapon 
system.  Specifically,  WLTA  uncovers  cues  that  initiate  task  behav¬ 
iors,  the  time  required  to  perform  the  tasks,  and  documents  various 
demands  the  tasks  place  on  the  individual.  Engineers  in  the  design 
of  weapon  systems  can  use  this  information  in  the  design  of  the 
weapon  system.  For  example,  in  the  ELINT  operator  example,  a 
software  modification  can  allow  the  workstation  to  “automatically 
identify”  targets,  thus  relieving  the  operator  from  the  require¬ 
ment  to  visually  identify  the  target.  This  information  helps  the 
training  analyst  establish  a  training  strategy  to  support  successful 
accomplishment  of  the  task.  For  this  reason,  two  training-focused 
performance  solutions  that  can  be  identified  and  implemented 
based  on  the  WLTA  data  are  what  to  train  and  how  to  train  specific 
tasks.  A  third  training  solution  that  can  be  derived  from  WLTA  data 
is  error  reduction.  Each  potential  solution  will  be  discussed  in  turn. 

What  to  Train 

The  benefit  of  the  information  provided  by  the  WLTA  is  that  it 
tells  training  analysts  where  operators  may  spend  most  of  their  time 
and  what  tasks  require  the  most  of  the  operator’s  limited  resources 
during  a  mission.  The  analysts  can  then  dig  deeper  into  this  informa¬ 
tion  to  understand  whether  these  are  areas  of  training  importance 
and  then  focus  training  on  that  area  of  tasks.  In  Figure  4,  a  deci¬ 
sion-making  matrix  that  could  be  used  by  instructional  systems 
developers  to  focus  training  is  provided.  In  this  contrived  example, 
reasonable  tradeoffs  of  what  to  train  can  be  made  to  focus  on  tasks 
that  require  High  Information  Processing  (I  and  III)  because  opera¬ 
tors  are  likely  to  require  the  most  support  in  performing  the  tasks. 
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Tasks  requiring  a  High  Time  Spent  but  Low  Information  Processing 
(II)  can  then  be  assessed  to  determine  whether  these  are  tasks  that 
could  be  automated  or  distributed  to  other  team  members  to  allow 
more  time  for  operators  to  perform  I  and  III  tasks.  Finally,  tasks 
within  the  IV  quadrant  may  be  identified  as  unnecessary  to  train, 
thereby  assisting  the  IPT  in  allocating  resources  for  the  best  return 
on  investment.  It  should  be  noted  that  this  decision-making  matrix 
is  intentionally  simplistic  for  the  purposes  of  this  article.  Factors 
such  as  the  criticality  of  the  tasks,  number  of  information  process¬ 
ing  channels  required,  and  others  important  to  the  weapon  system 
should  also  be  considered  in  determining  what  to  train.  In  the  ELINT 
operator  example,  an  early  decision  to  fund  a  software  modifica¬ 
tion  to  “automatically  identify”  targets  can  provide  life-cycle  cost 
reductions  from  a  training  perspective.  Since  targets  will  be  “auto¬ 
matically  identified”  by  the  workstations,  then  training  objectives 
relating  to  interpreting  the  information  will  be  incorporated  in  the 
curriculum.  Without  the  need  to  teach  how  to  manually  recognize 
the  targets,  the  course  can  be  shortened,  reducing  the  overall  life- 
cycle  training  costs. 


FIGURE  4.  DECISION-MAKING  MATRIX  TO  GUIDE  TRAINING 
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How  to  Train 

Once  decisions  are  made  regarding  what  to  train,  instructional 
systems  specialists  can  also  utilize  WLTA  results  to  determine  how 
best  to  train  the  skills  that  are  identified  as  training  tasks.  For  tasks 
that  fall  within  the  II  and  IV  quadrants  of  Figure  4,  low  information 
processing  is  required,  suggesting  that  these  tasks  are  relatively 
automatic  or  simplistic.  This  suggests  that  these  types  of  skills  may 
be  most  effectively  taught  partly  through  training  methods  such 
as  computer-based  online  courses  and  partly  through  task  training 
devices  or  training  simulators,  wherein  trainees  receive  a  demon¬ 
stration  of  how/when  to  perform  the  tasks  and  opportunities  to 
practice  performing  the  skills.  Conversely,  demanding  tasks  (i.e., 
high  information  processing  tasks  [I  and  III])  often  involve  more  cue 
complexity  and  mental  effort  (Wickens  &  Carswell,  2006).  With  this 
information  in  hand,  the  instructional  developer  can  appropriate 
more  time  to  train  complex  tasks  and  ensure  prerequisite  knowl¬ 
edge  and  skills  are  acquired  early  in  training.  Furthermore,  training 
strategies  can  be  chosen  to  ensure  more  trainee-instructor  interac¬ 
tion,  and  may  allow  operators  repeated  opportunities  to  practice 
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the  tasks  with  varying  and  increasing  levels  of  complexity  to  build 
the  decision-making  and  information  processing  skills  that  are  less 
outwardly  tangible  and  difficult  to  train. 

Reducing  Errors 

Interviews  with  SMEs  during  the  WLTA  often  reveal  common 
mistakes  (as  well  as  their  consequences)  made  by  operators.  Senior 
operators  will  comment  that  mistakes  in  performance  are  usually 
traceable  to  inattention,  over  attention,  or  fixation  (Greenwell, 
Strunk,  &  Knight,  2004;  Wickens  &  Carswell,  2006).  As  Carl  (2009, 
p.  120)  noted,  “...there  is  a  tendency  for  performers  to  devote  too 
much  time  to  some  cues,  devote  too  little  time  to  other  cues,  or 
poorly  manage  their  time  in  attending  to  all  the  cues  that  impact 
task  execution.”  With  this  information,  instructional  designers  can 
focus  on  initially  training  new  operators  to  select  and  concentrate 
on  important  task  cues  while  disregarding  irrelevant  noise.  In  this 
case,  utilizing  WLTA  data  will  not  only  emphasize  the  important 
components  of  training,  it  will  reduce  downstream  performance 
problems  that  will  increase  life-cycle  costs  for  repair  or  replace¬ 
ment  of  the  system.  In  the  ELINT  operator  example,  the  costs  for  a 
software  modification  to  “automate  target  recognition”  are  incurred 
only  once  during  acquisition  (unless  there  are  system  upgrades  in 
which  additional  costs  will  be  incurred).  Training  costs  associated 
with  teaching  target  recognition  will  reoccur  with  each  new  set  of 
trained  operators,  thus  affecting  overall  life-cycle  training  costs. 


Conclusions 

Although  acquisition  professionals  are  continually  asked  to 
identify  tradeoffs  to  reduce  weapon  system  budgets,  tools  are 
available  to  help  them  make  decisions  regarding  life-cycle  costs. 
Everyone  acknowledges  that  weapon  systems  being  acquired  today 
are  extremely  sophisticated.  The  operator  and  maintainer  tasks 
associated  with  the  weapon  systems  are  also  becoming  increasingly 
complex,  and  they  require  time  and  expensive  simulators  to  satisfy 
training  requirements.  Training  time  and  training  media  are  quite 
costly  from  a  life-cycle  perspective. 

WLTA  could  be  an  extremely  valuable  tool  in  reducing  life-cycle 
costs,  ensuring  the  system  can  be  used  effectively  by  designing 
training  that  can  support  the  operator.  Admittedly,  WLTA  is  no  sim¬ 
ple  task,  requires  significant  time  and  support  from  the  acquisition 
team,  and  must  be  adapted  to  the  needs  of  the  individual  program. 
However,  the  results  of  WLTAs  can  become  increasingly  valuable  as 
the  team  is  required  to  make  trade-off  decisions  and  must  negotiate 
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with  program  managers  to  retain  funding,  which  can  be  done  with 
the  support  of  systematically  derived  evidence  of  how  performance 
and  mission  requirements  will  be  affected  by  design  decisions. 
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Visual  Workload  Scale 

Scale  Value 

Visual  Scale  Descriptor 

0.0 

No  Visual  Activity 

1.0 

Visually  Register/Detect  (detect  occurrence  of  image) 

3.7 

Visually  Discriminate  (detect  visual  differences) 

4.0 

Visually  Inspect/Check  (discrete  inspection/static 
condition) 

5.0 

Visually  Locate/Align  (selective  orientation) 

5.4 

Visually  Track/Follow  (maintain  orientation) 

5.9 

Visually  Read  (symbol) 

7.0 

Visually  Scan/Search/Monitor  (continuous/serial 
inspection,  multiple  conditions) 

Auditory  Workload  Scale 


Scale  Value 

Auditory  Scale  Descriptor 

0.0 

No  Auditory  Activity 

1.0 

Detect/Register  Sound  (detect  occurrence  of  sound) 

2.0 

Orient  to  Sound  (general  orientation/attention) 

4.2 

Orient  to  Sound  (selective  orientation/attention) 

4.3 

Verify  Auditory  Feedback  (detect  occurrence  of 
anticipated  sound) 

4.9 

Interpret  Semantic  Content  (speech) 

6.6 

Discriminate  Sound  Characteristics  (detect  auditory 
differences) 

7.0 

Interpret  Sound  Patterns  (pulse  rates,  etc.) 
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Cognitive  Workload  Scale 

Scale  Value 

Cognitive  Scale  Descriptor 

0.0 

No  Cognitive  Activity 

1.0 

Automatic  (simple  association) 

1.2 

Alternative  Selection 

3.7 

Sign/Signal  Recognition 

4.6 

Evaluation/Judgment  (consider  single  aspect) 

5.3 

Encoding/Decoding,  Recall 

6.8 

Evaluation/Judgment  (consider  several  aspects) 

7.0 

Estimation,  Calculation,  Conversion 

Psychomotor  Workload  Scale 

Scale  Value 

Psychomotor  Scale  Descriptor 

0.0 

No  Psychomotor  Activity 

1.0 

Speech 

2.2 

Discrete  Actuation  (button,  toggle,  trigger) 

2.6 

Continuous  Adjustive  (Flight  control,  sensor  control) 

4.6 

Manipulative 

5.8 

Discrete  Adjustive  (rotary,  vertical  thumbwheel,  lever 
position) 

6.5 

Symbolic  Production  (writing) 

7.0 

Serial  Discrete  Manipulation  (keyboard  entries) 
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